Skip to content

Update Derecho site config #1906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
climbfuji merged 11 commits into from
Feb 11, 2026
Merged

Update Derecho site config #1906

climbfuji merged 11 commits into from
Feb 11, 2026

Conversation

@climbfuji
Copy link
Collaborator

@climbfuji climbfuji commented Feb 9, 2026 •
edited
Loading

Description

  1. Update Derecho site configuration to newer compiler versions, use native crayenv instead of ncarenv.
  2. Submodule pointer update for spack to fix an issue on Derecho - never attempt to recursively scan the root filesystem (/). See spack PR filesystem.py: prevent recursively traversing the root filesystem spack#576 for more information and links to upstream spack discussion.

Dependencies

Issues addressed

Working toward #1883

Applications affected

None

Systems affected

Derecho

Testing

  • CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
    • GitHub actions CI tests pass
    • GitHub actions CI tests do not pass (provide explanation)
    • GitHub actions CI tests skipped (provide explanation if necessary)
  • New tests added: List and describe any new tests added to GitHub actions
    • ...
  • Additional testing: Add information on any additional tests conducted
    • Built oneAPI and GCC stacks on Derecho (unified environment). Tried jedi-bundle: Compiled and ran tests (skipped MPI tests that required qsub) with GCC. Compiled and ran into the well-known error in MPAS with oneAPI (but cmake step completed and most of the compilation succeeded as well). This is as far as I can go.

Checklist

  • This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
  • These changes have been tested on the affected systems and applications.
  • All dependency PRs/issues have been resolved and this PR can be merged.
  • All necessary updates to the documentation (spack-stack wiki) will be made when this PR is merged

Wiki update for Derecho - will be made when PR is merged and spack-stack-2.1 user documentation is prepared

  1. How to build spack-stack environments. Nothing special to do, happens automatically when running source setup.sh
  2. How to use spack-stack environments:
# GENERAL

module --force purge

module use /opt/cray/pe/modulefiles
module use /glade/work/epicufsrt/contrib/spack-stack/derecho/installs/oneapi-2025.3.1/modulefiles


# ONEAPI

module use /lustre/desc1/scratch/heinzell/spst-20260209/envs/ue-oneapi-2025.3.1/modules/Core

module load crayenv/25.03
module load PrgEnv-intel/8.6.0

module load stack-intel-oneapi-compilers/2025.3.1
module load stack-cray-mpich/8.1.32

module unload cray-libsci

module available


# GNU

module use /lustre/desc1/scratch/heinzell/spst-20260209/envs/ue-gcc-13.3.1/modules/Core

module load crayenv/25.03
module load PrgEnv-gnu/8.6.0

module load stack-gcc/13.3.1
module load stack-cray-mpich/8.1.32

module unload cray-libsci

module available

@climbfuji climbfuji mentioned this pull request Feb 9, 2026
Merged
2 tasks
@climbfuji climbfuji self-assigned this Feb 11, 2026
@climbfuji climbfuji marked this pull request as ready for review February 11, 2026 02:20
@climbfuji climbfuji moved this to In Progress in spack-stack-2.1.x (2026 Q1) Feb 11, 2026
This was referenced Feb 11, 2026
Open
Merged
rickgrubin-noaa
rickgrubin-noaa approved these changes Feb 11, 2026
View reviewed changes
Copy link
Collaborator

@rickgrubin-noaa rickgrubin-noaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pleased to see that crayenv is available so as to be consistent with (from EPIC's perspective) gaea-c6.

A couple of comments for posterity:

  • oneapi@2025.3.1 is nice to have; its installation does place an onus on EPIC to maintain it, which is not necessarily desireable -- preference is for system tools to be installed by parties responsible for the system
  • a source cache is also nice to have, and has been discussed for other EPIC hosts. There are disk space constraints to keep in mind, as it's not always clear whether or not EPIC's allocations can be increased

@climbfuji
Copy link
Collaborator Author

climbfuji commented Feb 11, 2026

Pleased to see that crayenv is available so as to be consistent with (from EPIC's perspective) gaea-c6.

A couple of comments for posterity:

  • oneapi@2025.3.1 is nice to have; its installation does place an onus on EPIC to maintain it, which is not necessarily desireable -- preference is for system tools to be installed by parties responsible for the system

I would argue that maintaining this is so little work, maybe even less than dealing with what CISL throws at you.

  • a source cache is also nice to have, and has been discussed for other EPIC hosts. There are disk space constraints to keep in mind, as it's not always clear whether or not EPIC's allocations can be increased

I didn't add it, it was already there. I didn't populate it yet either. But happy to remove it if you think it's better.

@rickgrubin-noaa
Copy link
Collaborator

rickgrubin-noaa commented Feb 11, 2026

  • oneapi@2025.3.1 is nice to have; its installation does place an onus on EPIC to maintain it, which is not necessarily desireable -- preference is for system tools to be installed by parties responsible for the system

I would argue that maintaining this is so little work, maybe even less than dealing with what CISL throws at you.

I would argue that as well; I am obliged to represent EPIC mgmt's decisions and point of view.

  • a source cache is also nice to have, and has been discussed for other EPIC hosts. There are disk space constraints to keep in mind, as it's not always clear whether or not EPIC's allocations can be increased

I didn't add it, it was already there. I didn't populate it yet either. But happy to remove it if you think it's better.

I was not aware that the source cache was already there; given that, let's update it per the latest envs.

@climbfuji climbfuji merged commit 0860769 into JCSDA:develop Feb 11, 2026
6 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in spack-stack-2.1.x (2026 Q1) Feb 11, 2026
@climbfuji climbfuji deleted the feature/derecho branch February 11, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rickgrubin-noaa rickgrubin-noaa rickgrubin-noaa approved these changes

@RatkoVasic-NOAA RatkoVasic-NOAA Awaiting requested review from RatkoVasic-NOAA

Assignees

@climbfuji climbfuji

Labels

None yet

Projects

spack-stack-2.1.x (2026 Q1)
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@climbfuji @rickgrubin-noaa